Papers with bidirectional language model
Commonsense Knowledge Mining from Pretrained Models (D19-1)
Copied to clipboard
| Challenge: | Several approaches have been proposed for training models for commonsense knowledge base completion (CKBC) due to the sparsity of training data. |
| Approach: | They propose a method for generating commonsense knowledge using a large, pre-trained bidirectional language model by transforming relational triples into masked sentences. |
| Outcome: | The proposed method outperforms models trained on held-out test sets on a held-up set, suggesting that it generalizes better than current supervised methods. |
Like a Baby: Visually Situated Neural Language Acquisition (P19-1)
Copied to clipboard
| Challenge: | A multi-modal neural architecture outperforms its equivalent trained on language alone with a 2% decrease in perplexity . |
| Approach: | They propose to use visual context to train neural language models to perform next-word prediction. |
| Outcome: | The proposed model outperforms its equivalent trained on language with 2% decrease in perplexity even when no visual context is available at test. |
Modeling Long Context for Task-Oriented Dialogue State Generation (2020.acl-main)
Copied to clipboard
| Challenge: | Existing approaches to dialogue state tracking are limited to scenarios with infinite slot values and prediction of unseen slot values. |
| Approach: | They propose a multi-task learning model with a simple yet effective utterance tagging technique and a bidirectional language model as an auxiliary task for task-oriented dialogue state generation. |
| Outcome: | The proposed model achieves state-of-the-art accuracy on the MultiWOZ 2.0 dataset. |